Computer technology has been steadily advancing in recent years, with computers becoming ever more powerful, and computer languages becoming ever more sophisticated. This combination has resulted in the desire and opportunity to create more complex computer programs, particularly involving graphical user interfaces and human-computer interaction. The power of computers has been linked with more advanced languages to create a class of meta-applications, software tools which facilitate the creation of software programs. An example of such a tool is Microsoft Visual Studio, which enables drag-and-drop visual design of an application interface, with subsequent automatic or semi-automatic generation of code for the corresponding application. Another example is Macromedia Flash, which reduces the need for the author to write code, relying instead on a powerful (but limited) set of capabilities for specifying the response of the application to particular user inputs, then generating the corresponding code statements to make those responses occur when the input is received.
A characteristic of user-interface-based applications is that each application provides a limited set of mechanisms for user input. For example, a user may press a key on a keyboard, move the representation of a mouse pointer across the screen, or push a mouse button. Each of these ‘events’ can be utilized to instigate the execution of a particular sequence of code statements, which cause some change in the internal state, visual appearance or manifest behavior of the application. Stringing such events and responses together constitutes the utilization of the application, with some hopefully-useful larger result accruing from the accumulation of such steps.
Some kinds of interactive applications are typified by transitions among similar but discrete states of appearance. The application ‘window’ may for example display a set of buttons, one of which appears differently than the others by a color change (highlighting) or by a change in content (e.g., a tick mark overlaid on the button). As the user of the application provides input, the elements in the display change. As a typical example, the user might press a navigational arrow key, and in response, the appearance of the set of buttons will change so that the highlighting appears to move from one button to another. Such an action might be employed to allow the user to select among a number of mutually-exclusive options, and when the selection is finalized (indicated for example by the user pressing the ‘Return’ key), the program executes a particular behavior based on the selection.
The visual design tools that have been developed to create such applications solve a number of fundamental problems for the author. The tools often create the basic framework for the application, including the event recognition and dispatching code, and insert code statements to define the components which establish the initial visual appearance of the application on the computer screen, including for example, buttons, check boxes, text boxes with scroll bars, and so forth. Generally with such tools the appearance of the application can be statically defined, with limited automatic code generation for dynamic changes to the application.
An exception to this is the Macromedia Flash development tool. This tool allows for the definition of multiple states or appearances of the application, which can be selected by means of temporal sequencing, or in response to user input. An object or a collection of visual elements arranged on the screen is called a ‘frame’ or ‘scene’, and a frame can be displayed by a code statement in the ActionScript language of Flash. When an object such as a Button can have multiple states of behavior or appearance, each such state is represented by a discrete frame, and the selection of a particular frame enables control over the appearance and/or behavior of the object. Because ActionScript provides a full set of algorithmic capabilities, complex behaviors can be built up by the author. However, the display and appearance of multiple discrete objects must still be managed by an explicit linear sequence of code statements.
A static initial appearance can be defined within the context of the development tool, and the code (or data) which creates that appearance is automatically generated. In some tools, other static renderings of the scene, or of objects in the scene, may be created by the author using the tool, with automatic code or data generation. However, the selection among such static renderings must be manually established by the author. More subtle changes, which are not captured in static renderings, are exclusively under programmatic control, and must be manually generated.
The current technology for automatic generation of application appearance and behavior suffers from a second problem inherent in the approach described above. The Macromedia Flash development tool permits the description of multiple static frames for an object or collection of graphic elements. Some sequences of frames are shown in a fixed order—the author may define an animation consisting of a sequence of movements or transformations of graphic elements comprising a scene. In such cases, the animation may be efficiently encoded as a sequence of differences, or a sequence of algorithmic transformations, from one frame to the next. Such sequential encoding reduces the data requirements for storing the animation. For the frames which comprise a general object representation, however, this storage efficiency is lost because the transition among the various object frames is entirely arbitrary. In the course of the application, an object may change from one appearance mode to another, each mode being one of the statically-defined frames for the object. In order to display the newly-selected frame, all of the attributes of the frame must be available to enable the generation of the appropriate graphical appearance on the display, since (from the point of view of the development tool) the identity of the previous mode (and thus the value of each attribute in the frame) is arbitrary.
Consider the following description of an example interactive television application with reference to FIG. 1. Once the application is started, the viewer is presented with a display containing three buttons, one of which is highlighted. Pressing directional keys on the remote control causes the highlighting to move among the buttons. Pressing a special selection key on the remote control causes the application to display an image, where the content of the image is based on the button which was highlighted when the selection key was pressed.
The author might wish that pressing a certain remote control key, when the highlighting is on a particular button, causes highlighting to pass to another button according to the following table:
InitialhighlightedRemote control key pressed by viewerbutton ↓LeftRightUpDownButton 1Button 1Button 2Button 1Button 3Button 2Button 2Button 1Button 2Button 3Button 3Button 1Button 2Button 1Button 3
A conventional way of achieving this behavior is for the author to write explicit code statements to accomplish the required changes. For each state (that is, for each button which might be highlighted), the code statements would have to test for each possible remote control key, and depending on the actual key pressed, make changes to the properties of two buttons as required. The following pseudo-code fragment depicts one manner in which this might be accomplished:
FunctionToHandleButtonpress (int iKey)if ( Button1.Highlighted )if ( iKey == iKeyRight )Button1.Highlighted = FalseButton2.Highlighted = Trueelse if ( iKey == iKeyDown )Button1.Highlighted = FalseButton3.Highlighted = Trueelse if ( Button2.Highlighted )if ( iKey == iKeyLeft )Button2.Highlighted = FalseButton1.Highlighted = Trueelse if ( iKey == iKeyDown )Button1.Highlighted = FalseButton3.Highlighted = Trueelseif ( iKey == iKeyLeft )Button3.Highlighted = FalseButton1.Highlighted = Trueelse if ( iKey == iKeyRight )Button3.Highlighted = FalseButton2.Highlighted = Trueif ( iKey == iKeyup )Button3.Highlighted = FalseButton1.Highlighted = TrueReturn
If additional property changes were desired, such as moving the button, changing the displayed image, or hiding the buttons, more hand-written code would be required.
If the author wanted to change more than one parameter of a button, for each additional parameter to be changed, two statements would be required per if-else block. In this example, this would mean an additional 14 lines of code for each parameter to be affected.
Another approach to the automatic generation of behavior for this application, typified by the Macromedia Flash tool, is to describe a sequence of three frames, each of which contains one highlighted button and two non-highlighted buttons. Each frame would contain all the data required to describe the desired appearance. Then, for each frame, the effect of a button press would be to specify the next frame (which in some cases is identical to the current frame). This transition logic could be contained in a data table similar to Table 1 above. Upon a frame transition, all of the properties associated with that frame would be imposed on the elements of the application, regardless of whether those particular properties had changed from the previous frame or not. An event handler can then be structured to select among the possible frames, using a data matrix constructed from Table 1. This simplifies the event handler, but still requires that object values in each frame be defined one at a time.
Therefore, there exists a need to provide systems and methods to easily identify and code graphical or functional behavior states in an interactive application.